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Abstract 

New dynamical mechanism of quark mass generations and mixing 
is demonstrated in the examples of three and four generations. In the 
framework of the new mixing pattern, called the coherent mixing, the 
CKM elements are predicted compatible with experimental data for 
three generations, and are strongly constrained for four generations. 

PACS numbers: 12.15.Ff; 12.15.B0i; 12.60. Cu. 

1. The idea that quark masses can be obtained from composite scalar 
fields instead of elementary Higgs fields is actively developed last decades, 
for reviews and references see [U |2] . 

The basic underlying mechanism for the spontaneous electroweak symme- 
try breaking (EWSB) and mass generation was the effective quartic fermion 
interaction which is bosonized in the standard way. What is left unanswered 
in this approach is the dynamical origin of generations and the peculiar pat- 
tern of masses and mixings in the observed up to now three quark generations. 

It is a purpose of the present letter to describe, basing on the recent paper 
[3], the mechanism which produces three or more generations of quarks and 
can predict intervals of values for mixing coefficients and masses of t',b' in 
the case of the fourth generation. 

One starts with the unbroken gauge field (it can be taken as SU (N) 
field, but this is not important for what follows) and quark Lagrangian L = 
9i J a'y l iC^ b ^b } where fields are organized into two sectors ipa\i = 1,2. 
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«=(S)'(t)'(l) ; ^ =(M - 4 ' ta 

where if) c means the charge conjugated quark field. 

We assume that nonperturbative vacuum fields C M can form connected 
field correlators, such that (vacuum averaging of Euclidean fields is implied) 

tr < C^x^C^x^.-.C^n) J { ^M,x 2 , ...x n ). (2) 

We note, that the form (j2J) is in general not gauge invariant, but for non- 
confining field C M (x) one can define ([2]) in such a way, that in the local limit 
\xi~ Xj\ — > 0,i,j, = 1, ...n one can isolate in (T5]) a gauge invariant piece of the 
quark self-energy (this implies that the scale M of j( n ' can be taken much 
larger than any other mass in the problem). As a result one obtains in the 
partition function Z = J DCDipDipe 1 ; the effective quark Lagrangian 



(exp / Ld 4 x) = exp < J n (xi,...x n )^(xi)...^(x n )dxi...dx n >. (3) 

J Ln=2,4,6,... J 

Here ^>(x) = ^(xj^^ip^x), and we have suppressed spinor and group in- 
dices. 

Now making pairwise Fierz transformations and keeping only scalar and 
pseudoscalar terms, one can express all quark bilinears in terms of §rl and 
where $rl(xi,X2) = ^rOzOV'l^)) superscript a denotes £77(2) index 
of EW group or its GUT equivalent. Next one does a bosonization trick 
introducing auxiliary functions fj,(xi, Xk) , <p(xi, Xk) in 5-function terms and 
finding the stationary points of ([2]) in fi and ip 

(exp J ' Ld 4 x) c = J DpDp + 5(p - $ RL )5(p + - $ LR )x 



exp I - J n (^ + + (p + (f) n/2 dx 1 ...dx n > 



(4) 



where 5(f — F) = f Dfiexp {—if /«(/ — F)dx}. 

The final equation can be written in the momentum space for the quark 
mass function n(jp) in the form (see [3] for derivation and more discussion) 

A*(P) =Z~1 j D n{Pl> -Pn-l}Jn{p;Pl, -Pn-l}- (5) 
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Here D n {...} denotes product FJLi jffi d (Pk), d (p) = pqj^ ■ Note, that 
nip) plays the role of the p-dependent mass of quark, since it enters the quark 
Green's function as <5'~ 1 (p) = p + ifi(p). 

It is important, that the stationary points of the wave functional (jlj) can 
be written for the real quantity ip(xi,X)-) = —iip(xi,Xk) in the local limit 
(M — > oo) as follows 

(-□ + ^ = -f^f; u{tp} = - l -J^ + \j& - l -J^ + - (6) 

o<p[x) 2 4 o 

At this point one realizes that J n are positive for connected correlators and 
therefore U{(f>} can have multiple minima ipi. We can associate composite 
scalars of generations with these minima, and then solutions of ([H]) of kink 
type, which connect different minima i,k of U{fi} can be related to mixing 
solutions of composite scalars 0n~- The corresponding masses fin = fi{ and 
fiik can be found from the equation fi = (□ + fi 2 )(f> where both \x and (p are 
matrices in generation indices. In this way one obtains the initial mass matrix 
fiik, which should be diagonalized to yield the final physical mass matrix m 
and CKM mixing coefficients V^. We shall not make explicit at this point 
the coefficients J n and the functional [/{ft}, referring it to later publications, 
but rather shall try to guess the form of the matrix fi, corresponding to the 
realistic physical masses and CKM coefficients. 

Namely, because of the strong hierarchy of masses, /133 3> fin Mil, the 
matrix solution of ([5]) fiik acquires the approximate form fiik = y/jMf^k, with 
fii = fin and fik = fikk, and one can write a slightly distorted form 

fJ-ik = \Zfiifik(^ + Vik), i 7^ k; \rf ik \ < 1. (7) 

We shall call ([7]) as in [3] the Coherent Mixing Mechanism (CMM), and 
apply it to the case of three and four generations. 

2. In the case of three generations one can expand the eigenvalue equation 
det(/i — ml) = to the lowest orders in 77^ to find the physical masses rrii 

m 3 - m 2 a + m£ - ( = (8) 

with a = J2 3 r=1 fi h £ = - £ ¥i fnfijffij, C = (-| E^i Vij + 5 £ ¥i i+k VijVik 

For mi to be positive and for the natural hierarchy rri\ <C m,2 <C m.3, one 
needs 77^ < 0,i,/c = 1,2,3 and choosing r/ 13 = r/23 = —77, r/ 12 = —8, < 
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5^f], one has 



m 2 ~ — (/^i + /i 2 )77, m 3 w /ii + /i 2 + yu 3 - (9) 

m 3 

It is interesting, that for <5 1 the CMM has made the hierarchy much 

more pronounced, than original situation with \i\ < /^ 2 < M3) an d wii, m 2 can 
be made very close to zero, while is not far from ^3. 

The diagonalizing matrix W, defined as jl = W + mW, is given in the 
appendix 2 of [3] together with the resulting CKM matrix Vckm = W U W^. 

The matrix W can be conveniently written in terms of elements //^ and 
rrii, using the method of jl], in CMM this is simplified in the limit 8 — > 0, 
yielding condition W31 = 0, and an equivalent limiting form for the 4x4 
matrix W has three zeros: W31 = W41 = W43 = 

/ c a , -s a c p e i&1 \ s a s p e iSri \ 

W=\ -s a e- i5 ^, -c a c p ,c a spe i6 ™ (10) 

W is expressed via two sine parameters s a ,sp and two phases. Here the 
phases 8^ = Si — 8k, k — 1, 2, 3 satisfy conditions 

<*12 - 5i3 + <5 23 = 0. (11) 

Parameters c a ,cp,(si = \J 1 — cf , i = a,/3) are expressed via m^/i^-i = 
1, 2, 3 (similar expressions are found in jl]). 

/ m 2 - ill r, j / m 3 -^2 /: 2" 

C " = V^^i' gQ = V ^^' C/3= V 2m 3 -^- / x 3 ' "^V 1 -* 

(12) 

Surprisingly, the simple form (10) as will be seen yields realistic CKM 
matrix for three generations, which is written below in two ways: a general 
form, and another with assumption of 8% = 8f = for k ^ 1. 

V ud = clci + 44s u a s d a e^ + «44e iA « « £c£ + s^e iA " cos(/? u - 



Vus = slcyAe^-claie^+slsyAe^-^ « -c^fe^+^cf e i5 - cos(^-/5 d ) 
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V ub = s u a s u ^e iS ^ - sfclcy^ 5 ^ « s u a e 1 ^ sin(/3 u - fa) 
Vcd = slde-^ + c u n c u R c d R ste- l ^ + 



+ c^sisje**-*** « c«a^e- w » cos(/3 u - &) - a^e"*^ (13) 



T/ — S M „a! -iAi2 I u u d d , u d u d AA23 ~ „« _ei_-iA l a 1 „u „ci / o _#\ 
cs a a + C a C l3 C a C P + C a C a S li S P e ~ S a S a e + C a C a CO?, \Pu Pd)i 



Vtd = c^sje""" - c^sie-^-^ « s^e""" sin (/3 d - 

7 tt = s;4e" iA23 + c^cj « cos(& - 
Here Ay = 8§ - Sf j7 cos (3 U = dfc, cos/? d = cj. 

3. We compare now the CKM matrix elements with experimental data [5], 
first assuming three generations and adjusting values of fa, rrii in s^, si s^, si 
and Ai2,Ai3 to reproduce experimental bounds. 

For the CKM matrix (13) one can simplify replacing all cf , i = a, f3, s = 
u, d by unity (this amounts to (1 -j- 2)% accuracy). 

One takes values of m ; at the scale 2 GeV from [5], m u = 2 MeV, = 5 
MeV, m c = 1.25 GeV, m s = 95 MeV, m b = 4.54 GeV, m t = 173 GeV. 

^From (TlBl one obtains 



0.485 ±0.08, ^ = = 0.21 ±0.03, (14) 



V td ' K.I 



a« = 1^*1 = 0.095 ±0.012, 



cb\ 



and the last two values are compatible with the first one. Eqs. ( fl4l) . ( Fl2l) 
allow to derive /i M = 13 MeV, /i d = 9 MeV. 
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Next is the case of s^,Sp which always enter in ( I13p with the relative 
phase A 23 , 

\Vd,\ = \V U \ = \4 ~ e~ iA23 s u p\ = 0.0412 ± 0.0011. (15) 

We assume at this point, that A 2 3 = (which yields A 12 = Ai 3 )0 and ( IT5|) 
is satisfied using ( TT2"|) with p s = 0.29 GeV and fi c = 12.25 GeV. This yields 
s'p = 0.252, Sp = 0.211. Finally, fit, are defined from the CMM condition 
im « 160 GeV, fi b = 4.2 GeV. 

One can now check all construction computing reparametrization invari- 
ant quantities: angles a,f3,j, Wolfenstein parameters p, fj [6] and Jarlskog 
parameter J [TJ. 

FromdU a = arg (-^-) = A 12 = (99^ 3 )°[5], while 

\ ub u d / 

VcbVcd \ „„/„ d « -iAi 



/? = arg ( J = arg( S * - £e-*»). (16) 

Taking central values of s^,s^ from (jl4"|) and a in the experimental 
bounds, one obtains 0.67 < sin 2/3 < 0.8, which is compatible with PDG 
value sin 2/3 = 0.687 ± 0.0324. Another check is calculation of p + ifj [6j from 
f[T4"|) and experimental value of a = A 12 , 



VudV: b = - sje -^ ' 



p + ifj = = ° v Q ° .' = 0.208 + ^0.337. (17) 



This value is well compared with experimental (p + if)) exp = 0.221+Q;Q 2 g + 
z0.34^q;q45. Finally the Jarlskog parameter for central values from ( TT3"]) . f }T4l 
is 

J = sls*\sj - sy A2[i \ 2 sin A12 = 3.15 • HT 5 (18) 

vs experimental j5j ./ (3.08±g;if) • 10" 5 . 

We thus see, that CKM matrix (JTSj) provides a simple and reasonable 
form in good agreement with experimental data for three generations. 

The same is true also for the neutrino mixing matrix V eu , and it is demon- 
strated in Appendix 2, that V ev acquires the tribimaximal form, when masses 



1 This assumption implies that the CP violating phase is generated by the interaction 
of lowest generation with the nontrivial vacuum of the field C M ,and will be discussed at 
length elsewhere. 
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fa have reasonable values and m; satisfy experimental bounds. The differ- 
ence from the quark matrices lies in a very small value of — > 0, ( while 

~ 0.2), which leads to the tribimaximal form of V eu . 

As it is, one can persuade oneself, that the last form of the CKM matrix in 
f fT3|) is defined by three angles and one phase, e.g. by s%, and sin( ) 5 n — fid) = 
s^dp — Spdp, and the phase A 12 = a. The three angle parameters are 
expressed via fa, fa and one combination of (fa, fi s ) (for sin((3 u — (3d)). This 
corresponds to the actual number of parameters in the 3x3 unitary matrix, 
but in our case all parameters have a clear physical meaning. 

4. We turn now to the case of four generations. 

First of all one realizes, that the Pagels-Stokar relation [S],[T] for our mul- 
tiple stationary solutions fa(p) of Eq. (J5J) analyzed in [3J, has a generalized 
form 

( 19 ) 

^ i=l ^ 

where i> = 246 GeV and fa = fa(p) is the effective value of fa(p) in the 
corresponding loop integral. Since one assumes that M 3> max fa = fa the 
value of fax is limited from above and for M > 4 fa one has fa < 0.8 TeV. 
Now one can connect fa and m, in the same way, as it was done above for 
three generations (see appendix 3 of [3] for details). One has 

^4 = y + y^-«2, m 3 = — - W— - a 2 ,ai = 2^Mi> ( 20 ) 

a 2 = fa(fa + fa)v + 0(r), 6). 

here fj = —r) 34: > 0, and if fj ^> \r), then relations (J5]) for mi, are approx- 
imately the same, so that we keep the values fa,fa,jx u ,fa of the previous 
section. 

We take two estimates for fxf. a) fa\ = 0.8 TeV, ^ = 0.4 TeV; 
b) fa\, /ig = (0.5, 0.3) TeV and obtain fj±, s\ in two cases. 

a) 774 = 0.55, S4 = 0.51; = 0.7, s\ = 0.54. For faj, fa\ the situation 

is less constrained, since one can choose fif <C fa\ and one has sf -C 1, fjd "C 1 
and an equivalent limiting form for the 4x4 matrix W has three zeros: 
W 3 i = W41 = W42 = 
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w 



( c a 

— < 

o, 



-s a cpe 

-C a Cp, 

0, 



«<5l2 



c 4 s a Sf3e 
c 4 spc a e 

C 4 Cp, 

-s A e~ i534 , 



s 4 sps a e L 
s A s p c a e 
s 4 cpe i5:ii 
c 4 



■ld-24 



(21) 



Here the phases 5^ satisfy conditions 



#24 = ^23 + <534, £14 = ^13 + #34, #14 ~ ^24 = #12- (22) 

The 3x3 matrix W is easily obtained putting s 4 = 0, C4 = 1. 
Parameters c a , c^, c 4 (si = \J\ — cf, i = a, /?, 4) are expressed via mi, fii,i = 
1,2,3,4 



2 _ m 2 - /xi 2 _ m 3 - fi 2 2 a, ^4 ~ m 3 + jtf 2 , OQ ^ 

C Q — , Co — , C4 — . ( ) 

m 2 — mi M m 3 + yUi - mi - m 2 m A - m 3 + jj, 2 

The resulting CKM matrix is given in Appendix 1. 

One can check, whether CKM matrix is compatible with experiment in 
the case of four generations. The sensitive points are unitarity relations and 
the matrix element V u d- 

We list as an example two possible sets of masses \Xi which define together 
with rrii the parameters s\, c l k , i = u,d; k = a, (3,4 

H d = 10 MeV, fi 8 = 0.29 GeV, fi u = 13 MeV, // c = 12.25 GeV; 
set a) fit, \i t < = (400, 800) GeV; fx b , (i v = (20, 400 GeV); m v ,m t , = (420, 1200) 
GeV; 

set b) fi t , fjtff = (300, 500) GeV; fi b , fi v = (200, 380) GeV; m v ,m t , = (580, 630) 
GeV. 

The resulting parameters are: 
s u p, = 0.252; 0.968; s^; c u a = 0.095; 0.995 
sf, cj = 0.211; 0.98; s d a , c d a = 0.21; 0.98; 
a) s «; c\ = 0.51; 0.86; sj; cj = 0.22, 0.97; 
h)sl, c\ = 0.54; 0.85; sj; cj = 0.57, 0.82. 

Let us first check, whether the 3x3 part of the CKM matrix is not 
deteriorated by the inclusion of the 4-th generation. 

Taking V u d and parameter values from Appendix 1, one can write 

V ud = 0.975 + 0.019 • e iAl2 + 0.00084e iAl3 + 1.2 • 10~V Al4 . (24) 
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Assuming as before, that the nonzero phase is due to the first generation 
only, i.e. A 12 = A 13 = A 14 = a and a in experiment is close to | [5], one 
obtains a good agreement with experimental value for V u d, \V u d\ = 0.97418 ± 
0.00027 [5] with strong limits on a, \a - 90° | < 1.5°. 

Another CKM element known with good accuracy is V us and with the 
same parameters from Appendix 1 one obtains for A 12 = a = 90°, and 
the same assumptious about A ik = with i,k ^ 1, |V^. S | = 0.228, which 
is close to the experimental value [5]: \V US \ = 0.2255 ± 0.0019, and \V c d\ = 
0.215 vs experimental \V c d\ = 0.230 ±0.011 15] . In a similar way one can check 
other CKM coefficients which are known experimentally with larger errors 
and therefore this check is not sensitive to the contribution of the fourth 
generation. 

5. At this point one can compare our results with that in the literature, 
for reviews and references see [9], [10] . 

First of all, we note, that our 4x4 CKM matrix contains two additional 
phases and two angles (s^, sf) as compared to the 3x3 matrix. Moreover, in 
the one-phase limit (A 12 = A 13 = A 14 , A ik — 0, i, k ^ 1) no additional phase 
appears, and s 4 , sf and hence all V^j are defined by the masses /i 4 ' d , m% , in 
addition to four parameters of 3 x 3 matrix, and the resulting scheme is rather 
rigid with six overall parameters, which is less than 9 possible parameters of 
4x4 unitary matrix (with 2n — 1 phases removed). 

First of all, it is important to compare our results with the bounds [TT] - 
[14] on the mixing coefficients and masses my , m t > , following from FCNC box 
diagrams and the decay b — > sj, and also from the so-called precise EW tests 
(EWPT). The FCNC bound on mixing coefficients Vib',V t 'i, were studied in 
[14] . and can be written in our terms (assuming as before the only nonzero 
CPV phase An = a) as the bound on the combination K = — c%sf + s\c\ < 
0.6 (for a "conservative bound" [H]). From our mixing coefficients above 
one has K = 0.3 for the set a) and K = 0.042 for the symmetric set b). 
Thus both sets satisfy the FCNC bounds of [TS] . However, as shown in [IB] , 
the EWPT bounds are much more restrictive, and can be again reduced to 
the bound on the same quantity K{= s 34 in notations of [13]), e.g. K < 0.1 
(95% C.L.). One can see, that this limit is easily satisfied by our set b), 
where masses my ~ m t i ^0.6 TeV and me — my ~ 50 GeV, in the same 
range, as the values considered in [121 EE]- Thus one can see, that the mixing 
Vty ~ K pd —Vtfb ~ 0.1 can be easily accommodated in the coherent mixing 
scheme, and it satisfies both FCNC and EWPT bounds. On another hand, 
as it is stressed in [T5] , this mixing opens up an interesting possibility of the 
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search for the 3-body decay of Higgs in processes like H — > i'bW + or b'tW" , 
where the wide composite Higgs is in the TeV region. 

Summarizing, a new mechanism producing several generations of quarks, 
and mixing between them is suggested. The resulting CKM matrix in agree- 
ment with all experimental data is found for three generations. In the case of 
four generations stringent FCNC and EWPT bounds on masses and mixings 
drastically limit available space of mass parameters. It is also interesting, 
that our scheme provides a simple acceptable parametrization of the lepton 
mixing matrix, as shown in Appendix 2. 

As it was stressed recently in [16], in the general situation with m v i ^ 
\mz>, the case of four generations is still disfavored in the global fit analysis, 
including oblique parameters S, T, U and FCNC constraints. For possible 
implications of the latter see [T7] . 

Our scheme predicts the EWSB connected primarily with b, t quarks in 
case of 3 and with the b',t' quarks in case of four generations (see e.g. Eq. 
(JT9l)). and this is in common with approaches derived in [IS] . 

The author is grateful for useful discussion to V.A.Novikov, V.I.Shevchenko 
and M.I.Vysotsky. The author acknowledges financial support of the RFFI 
grant 09-02-00629a. 
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The CKM 4x4 matrix elements 

The CKM matrix V = W U W^ is obtained from (11 01) with parameters 
s^, Sp, s 4 , s^, s d p, sf (and the corresponding cosine terms), and phases 8%, Sfj, 
satisfying conditions ffTTj) and fl22|) . 

T/ , _ n u d 1 „u d u d iA l2 1 Q u r d Q d c d e iA 13 _i_ e « <-,d e « e ^ e d ^ A i4 . 

Vud — <- Q C Q -T b a b a LpCpt. -f C A b a b pC^b pb a d -f b 4 b 4 b pb a bpb a t 

V cd = -slcie- iS ™+cZcfoi4e- iS ^ 

(Al.l) 

T/ _ a u „d -»Ai2 I u u d d 1 „u d u u d d iA 23 , „u d u u d d iA 2i 
Vcs — i> a b a ti "I" t a tpt a tp "I- C 4 C 4 bpC a bpC a ti -|- b A b±bpC a bpC a t. 

T/ _ „M <i — iA23 1 „u d u d , c «„(i « d 1A34 
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v tv = -c\sicy^ + s u 4 4cy s ^ 

V Vd = -slc d A s d a s d p e- 1 ^-^ + c u A sis d s d a e-^ 
V t >s = -s u 4 c d s d c d Q e- i5 ^- iS ^ + c u 4 s d s d c d a e- l < 
V vh = -slcic d e- i5 ^ + cls d 4 c d e- 1 ^ 

V vv = s u 4 sje- iA3 * + c u 4 c d 4 

Parameter values s^, s*, Sp, sp 1 were discussed in the text, Eq. fl23l) and 
below. 



Appendix 2 

Coherent mixing in leptons 

In this appendix we apply the CMM matrices to families of e- and v- 
sectors. As in the main text and in [3] we define the W e , W v matrices in the 
following way 

Ae,v = W+ v m e>v {diag)W e<v (A2.1) 

so that the mixing matrix is V e<v = W e W+. 

In the e-sector with masses m e = 0.51 MeV, m M = 105.66 MeV and 
m T = 1777 MeV. 

Choosing /i e = 1 MeV, and not fixing yet and fj, T , one obtains the 
matrix W e in the simplified from ({TO]) , where we put c a — > c e a , s a —> s e a1 cp — > 
c%, sp — > —s e p. We shall neglect for simplicity possible Majorana phases, and 
obtain using (1T2"1) 

^ = 0.07^0, c£S*l; s% eg. (A2.2) 
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In the same way we are obtaining the matrix W v as in ( I12p replacing sp in 
( TTUj) by (+Sp), and we have 



-+0) = ( -<cos(/3 e + /3,)e-^ 
-s v a sm((3 e + f3 v )e- iS ™ 



C cos(/3 e + /?„) 
< sin(/3 e + 





- sin(/? e + /3„ 
cos(/3 e + /3„) 

(A2.3) 

where cos/? e = c|, cos ^ = c^, and using experimental data [5], sin6> 13 < 
5.1(T 2 , sin 2 (2# 23 ) > 0.90, 



-73' 
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0.471^, one has 



(A2.4) 



In this way one reproduces the well-known tribimaximal matrix V ev (s e a — > 
0) = Utb, where 

/ & 



V 



3 
1 

t 



l 
i 

f 

V3 



\ 
1_ 



1 

V2 



(A2.5) 



At this point one should find /i T and the set i = 1,2,3, which cor- 
respond to the conditions flA2.4j) . Having in mind, that = \^ T Sj ~r^ ~ 

f&L > 4 ~ 0.25 and s u p > « 0.42, (for experimental values [5] 



m 2 



0.875-10 eV, 771,3 ~ 4.9-10 eV, and assuming 7712,3 3> mi) one has 
a narrow interval, j3 e > 15°, > 25°, /3 e + = 45°, and \i v z varies in the 



interval 0.75?7i3 < /jg < 0. 827773 



777, 



< < 2m^. For the first generation 



of v both n\ > mi should only be much smaller, than 777 2 , 7773. 

For more discussion with the use of W matrices of the type of Eq. (1101) 
see [I]. 
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